1. Technical Field
The technical field relates to an imaging apparatus and a flash determination method, applied in a digital camera, video camera, or the like that captures continuous images, that does not cause the occurrence of band-shaped noise even when there has been an outside flash of light and that is capable of appropriately outputting a frame image at the instant of the flash of light (the time when the outside flash of light has occurred).
2. Description of the Related Art
Video cameras that capture images using CMOS (Complementary Metal-Oxide Semiconductor) image sensors (CMOS image sensors) are increasing as of late. With such a video camera that employs a CMOS image sensor, in the case where there has been an outside flash of light caused by a flash illumination or the like during the capturing of a moving picture, the entire screen does not become bright for the instant corresponding to a single frame; instead, an unnatural image in which the brightness differs between an upper area and a lower area of the screen is outputted. This problem is generally known, as indicated by the descriptions in, for example, Patent Citation 1 (JP2007-306225A), FIG. 3, paragraphs [0059] to [0065].
A drawing will be used to illustrate the cause of an unnatural image in which the brightness differs between an upper area and a lower area of the screen being outputted by an imaging apparatus that employs a CMOS image sensor.
FIG. 7 is a diagram illustrating an exposure timing when an image with a frame frequency of 60 Hz and 1,080 effective scanning lines has been captured using an imaging apparatus that employs a CMOS image sensor, and illustrates the occurrence of a band-shaped high-luminance area when there has been an outside flash of light. With a CMOS image sensor, the exposure timing is shifted from line to line, and thus there is a difference of approximately 1/60 of a second between the first line in the uppermost area of the screen and the 1,080th line in the lowermost area of the screen. Here, when there is an outside flash of light, the influence of that outside flash of light occurs across the lines exposed as the image of a preceding frame (2) and the lines exposed as the image of a following frame (3). As a result, a frame in which the lower area of the screen is bright occurs and is followed by a frame in which the upper area of the screen is bright in the image (video) outputted from the imaging apparatus. Meanwhile, because the outside flash of light occurs asynchronous to the operation of the imaging apparatus, it is unclear in which line the screen will start to become bright in the image outputted by the imaging apparatus (the captured image).
In response to this problem, where an unnatural image in which the brightness differs between an upper area and a lower area of the screen in the captured image, Patent Citation 1 proposes a method that detects an outside flash of light and, when the outside flash of light has been detected, recording is carried out excluding an unnatural image that has been affected by the outside flash of light.
Meanwhile, an imaging apparatus disclosed in Patent Citation 1 detects an outside flash of light in the manner described hereinafter. The imaging apparatus disclosed in Patent Citation 1 includes a detection unit that detects a brightness, a color distribution, or the like in an image. Information indicating the brightness, the color distribution, or the like in a predetermined area of the image, information indicating a spatial frequency of a predetermined area of the image, or the like is outputted as an evaluation value, and the difference between the evaluation value and an evaluation value of an image for comparison is taken; if the difference is greater than or equal to a reference value, it is determined that an outside flash of light is acting on the image.
However, with video cameras used in news coverage and the like, there are cases where one wishes to record an image at the instant of a flash illumination as well, such as outdoor nighttime scenes where a person is being transported in an automobile as part of a police escort.
In other words, there are cases where, when capturing images with a video camera that employs a CMOS image sensor, one wishes to output a uniformly bright image at the instant of a flash, rather than an unnatural image in which the brightness differs between the upper and lower areas of the image, even when there has been an outside flash of light.
However, when detecting an outside flash of light, the accuracy of the detection poses a problem.
In Patent Citation 1, brightness information of a predetermined area is used as an evaluation value, and the difference between that evaluation value and the evaluation value of an image for comparison is taken; if that difference is greater than or equal to a reference value, a flash detection process is carried out assuming that an outside flash of light is acting on the image. However, there are no discussions as to what specific region within the screen is to be used as the target of the flash detection process. Here, the accuracy of flash detection will be discussed for two cases, or (1) the case where the entire screen is taken as the region to be monitored and the average luminance of the entire screen is employed as the evaluation value, and (2) the case where only a single line in the lowermost area of the screen is taken as the region to be monitored and the average luminance of that line in the lowermost area is employed as the evaluation value.
(1) First, the accuracy of flash detection for the case where the entire screen is taken as the region to be monitored and the average luminance of the entire screen is employed as the evaluation value will be discussed. In the case where imaging is carried out by an imaging apparatus in an environment in which a flash has occurred, the average luminance also increases in the screen in a captured image obtained using the imaging apparatus; however, there are large variations in the increase of this average luminance. As illustrated in FIG. 7, because an outside flash of light occurs asynchronous to the operation of the imaging apparatus, it is unclear in which line the screen will start to become bright in the captured image. As a result, there are cases in which the screen in the captured image brightens starting with an upper area, and cases in which the screen in the captured image brightens from below.
FIGS. 8A and 8B are diagrams illustrating an example in which a screen in a captured image, whose luminance level is at a uniform 20%, experiences an outside flash of light, resulting in the lower area of the screen rising to a luminance level of 100%. FIG. 8A illustrates an example in which 90% of the screen experiences an increase in luminance due to the outside flash of light, whereas FIG. 8B illustrates an example in which 10% of the screen experiences an increase in luminance due to the outside flash of light. Note that “a luminance level of 100%” refers to a level that corresponds to a white peak level as defined by SMPTE (Society of Motion Picture and Television Engineers), the Association of Radio Industries and Businesses Studio Standard, and so on. The level of an image signal that contains a luminance level (a signal value) is expressed as a percentage of this white peak level, which is taken as “100%”.
In FIG. 8A, the average luminance for the entire screen is calculated as 92%. Meanwhile, in FIG. 8B, the average luminance for the entire screen is calculated as 28%. In other words, if the average luminance of the entire screen is calculated and taken as the evaluation value even when there is an outside flash of light, the evaluation value (in this case, the average luminance level of the entire screen) will vary greatly, from a level of 28% to a level of 92%, depending on where the line of the high-luminance area starts.
Here, if the reference value for determining a flash in the aforementioned flash detection method is set to a value that is twice the average luminance level of one frame (frame image) previous, according to the aforementioned technique, a flash will be determined to have occurred in the case where the average luminance level of the current frame image (the frame image to be processed) is greater than or equal to 40%. In the case of FIG. 8A, the evaluation value is 92%, and thus a flash can be determined to have occurred according to the aforementioned flash detection method; however, in the case of FIG. 8B, the evaluation value is 28%, and thus a flash is not determined as having occurred according to the aforementioned flash detection method.
Meanwhile, if the reference value for determining a flash is lowered in order to reduce the influence of the first line of high luminance, such as setting the reference value to 1.2 times the average luminance level of the frame (frame image) one frame previous, the average luminance level in the frame image is greater than or equal to 24%, and thus the aforementioned flash detection method determines that a flash has occurred. At this time, according to the aforementioned flash detection method, a flash can be determined to have occurred in both the cases illustrated in FIGS. 8A and 8B; however, the luminance in the screen changes in the captured image (the frame image) during normal imaging, when no flashes occur, due to zoom operations, pan/tilt operations, or when the subject moves, and there is thus the possibility that a flash will mistakenly be detected as having occurred.
As described thus far, with a technique that employs the average luminance of the entire screen in a captured image (a frame image) as the evaluation value and monitors changes in the evaluation value, there are situations where there is no response and no flash is detected as having occurred even when there has been a flash, and situations where a flash is mistakenly detected as having occurred even through no flash has actually occurred.
(2) Next, the accuracy of flash detection of a flash detection method will be discussed for the case where, as opposed to the case described in (1) above, the monitored region is reduced, taking a single line in the lowermost area of the captured image (the frame image) as the monitored region, and employing the average luminance of the single line in the lowermost area as the evaluation value.
In this case, the average luminance of the single line in the lowermost area changes significantly due to movement of the video camera. For example, when a tilt operation is performed in the video camera or the video camera is shaken, a subject (object) that had been captured in the lowermost area of the screen in the captured image (the frame image) moves in the vertical direction, and thus the average luminance of the single line changes greatly. FIG. 8C illustrates an example in which a high-luminance object has entered the frame, moving in an upward direction from the bottom of the screen in the captured image. In the case of FIG. 8C, the average luminance of the line in the lowermost area of the captured image rises suddenly from a level of 20% to a level of 50% between the two frames. In FIG. 8C, the evaluation value of the frame image one frame previous to the current frame is a level of 20%, and thus if, in the flash detection method, the reference value for determining whether a flash has occurred is set to twice the value of the average luminance level of the line in the frame image one frame previous, the average luminance level of the line in the current frame is greater than or equal to 40%, and a flash is determined to have occurred as a result. In other words, in the case of FIG. 8C, the aforementioned flash detection method mistakenly detects a flash as having occurred despite the fact that no outside flash of light actually occurred.
As described thus far, the conventional technique is problematic in terms of the detection accuracy when detecting a flash of light, in that mistaken detections and non-responses occur depending on the region that is set to be monitored for changes in luminance in order to detect flashes.
Having been achieved to solve the aforementioned conventional problem, it is an object of the present invention to achieve an imaging apparatus, a flash detection method, and a recording medium in which a program is recorded, that increase the detection accuracy of an outside flash of light.